{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "报社等相关的机构，往往会遇到一个问题，就是别人家的机构使用自己的文章但是并没有标明来源。将解决新华社的文章被抄袭引用的问题。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import re\n",
    "import time\n",
    "import os\n",
    "import joblib\n",
    "import numpy as np\n",
    "import pandas as pd\n",
    "from scipy.sparse import save_npz, load_npz\n",
    "import jieba\n",
    "from sklearn.feature_extraction.text import TfidfVectorizer\n",
    "from sklearn.model_selection import train_test_split, GridSearchCV\n",
    "from sklearn.neighbors import KNeighborsClassifier\n",
    "from sklearn.svm import SVC\n",
    "from sklearn.naive_bayes import GaussianNB\n",
    "from sklearn.linear_model import LogisticRegression\n",
    "from sklearn.ensemble import RandomForestClassifier\n",
    "from sklearn.metrics import precision_recall_curve, precision_score, recall_score, f1_score, accuracy_score"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 预处理原始文本"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "89611\n",
      "78661\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>id</th>\n",
       "      <th>author</th>\n",
       "      <th>source</th>\n",
       "      <th>content</th>\n",
       "      <th>feature</th>\n",
       "      <th>title</th>\n",
       "      <th>url</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <td>0</td>\n",
       "      <td>89617</td>\n",
       "      <td>NaN</td>\n",
       "      <td>快科技@http://www.kkj.cn/</td>\n",
       "      <td>此外，自本周（6月12日）起，除小米手机6等15款机型外，其余机型已暂停更新发布（含开发版/...</td>\n",
       "      <td>{\"type\":\"科技\",\"site\":\"cnbeta\",\"commentNum\":\"37\"...</td>\n",
       "      <td>小米MIUI 9首批机型曝光：共计15款</td>\n",
       "      <td>http://www.cnbeta.com/articles/tech/623597.htm</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>1</td>\n",
       "      <td>89616</td>\n",
       "      <td>NaN</td>\n",
       "      <td>快科技@http://www.kkj.cn/</td>\n",
       "      <td>骁龙835作为唯一通过Windows 10桌面平台认证的ARM处理器，高通强调，不会因为只考...</td>\n",
       "      <td>{\"type\":\"科技\",\"site\":\"cnbeta\",\"commentNum\":\"15\"...</td>\n",
       "      <td>骁龙835在Windows 10上的性能表现有望改善</td>\n",
       "      <td>http://www.cnbeta.com/articles/tech/623599.htm</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>2</td>\n",
       "      <td>89615</td>\n",
       "      <td>NaN</td>\n",
       "      <td>快科技@http://www.kkj.cn/</td>\n",
       "      <td>此前的一加3T搭载的是3400mAh电池，DashCharge快充规格为5V/4A。\\r\\n...</td>\n",
       "      <td>{\"type\":\"科技\",\"site\":\"cnbeta\",\"commentNum\":\"18\"...</td>\n",
       "      <td>一加手机5细节曝光：3300mAh、充半小时用1天</td>\n",
       "      <td>http://www.cnbeta.com/articles/tech/623601.htm</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>3</td>\n",
       "      <td>89614</td>\n",
       "      <td>NaN</td>\n",
       "      <td>新华社</td>\n",
       "      <td>这是6月18日在葡萄牙中部大佩德罗冈地区拍摄的被森林大火烧毁的汽车。新华社记者张立云摄\\r\\n</td>\n",
       "      <td>{\"type\":\"国际新闻\",\"site\":\"环球\",\"commentNum\":\"0\",\"j...</td>\n",
       "      <td>葡森林火灾造成至少62人死亡 政府宣布进入紧急状态（组图）</td>\n",
       "      <td>http://world.huanqiu.com/hot/2017-06/10866126....</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>4</td>\n",
       "      <td>89613</td>\n",
       "      <td>胡淑丽_MN7479</td>\n",
       "      <td>深圳大件事</td>\n",
       "      <td>（原标题：44岁女子跑深圳约会网友被拒，暴雨中裸身奔走……）\\r\\n@深圳交警微博称：昨日清...</td>\n",
       "      <td>{\"type\":\"新闻\",\"site\":\"网易热门\",\"commentNum\":\"978\",...</td>\n",
       "      <td>44岁女子约网友被拒暴雨中裸奔 交警为其披衣相随</td>\n",
       "      <td>http://news.163.com/17/0618/00/CN617P3Q0001875...</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "      id      author                  source  \\\n",
       "0  89617         NaN  快科技@http://www.kkj.cn/   \n",
       "1  89616         NaN  快科技@http://www.kkj.cn/   \n",
       "2  89615         NaN  快科技@http://www.kkj.cn/   \n",
       "3  89614         NaN                     新华社   \n",
       "4  89613  胡淑丽_MN7479                   深圳大件事   \n",
       "\n",
       "                                             content  \\\n",
       "0  此外，自本周（6月12日）起，除小米手机6等15款机型外，其余机型已暂停更新发布（含开发版/...   \n",
       "1  骁龙835作为唯一通过Windows 10桌面平台认证的ARM处理器，高通强调，不会因为只考...   \n",
       "2  此前的一加3T搭载的是3400mAh电池，DashCharge快充规格为5V/4A。\\r\\n...   \n",
       "3    这是6月18日在葡萄牙中部大佩德罗冈地区拍摄的被森林大火烧毁的汽车。新华社记者张立云摄\\r\\n   \n",
       "4  （原标题：44岁女子跑深圳约会网友被拒，暴雨中裸身奔走……）\\r\\n@深圳交警微博称：昨日清...   \n",
       "\n",
       "                                             feature  \\\n",
       "0  {\"type\":\"科技\",\"site\":\"cnbeta\",\"commentNum\":\"37\"...   \n",
       "1  {\"type\":\"科技\",\"site\":\"cnbeta\",\"commentNum\":\"15\"...   \n",
       "2  {\"type\":\"科技\",\"site\":\"cnbeta\",\"commentNum\":\"18\"...   \n",
       "3  {\"type\":\"国际新闻\",\"site\":\"环球\",\"commentNum\":\"0\",\"j...   \n",
       "4  {\"type\":\"新闻\",\"site\":\"网易热门\",\"commentNum\":\"978\",...   \n",
       "\n",
       "                           title  \\\n",
       "0           小米MIUI 9首批机型曝光：共计15款   \n",
       "1     骁龙835在Windows 10上的性能表现有望改善   \n",
       "2      一加手机5细节曝光：3300mAh、充半小时用1天   \n",
       "3  葡森林火灾造成至少62人死亡 政府宣布进入紧急状态（组图）   \n",
       "4       44岁女子约网友被拒暴雨中裸奔 交警为其披衣相随   \n",
       "\n",
       "                                                 url  \n",
       "0     http://www.cnbeta.com/articles/tech/623597.htm  \n",
       "1     http://www.cnbeta.com/articles/tech/623599.htm  \n",
       "2     http://www.cnbeta.com/articles/tech/623601.htm  \n",
       "3  http://world.huanqiu.com/hot/2017-06/10866126....  \n",
       "4  http://news.163.com/17/0618/00/CN617P3Q0001875...  "
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "filename = 'datasets/news_from_where.csv'\n",
    "rawdata = pd.read_csv(filename, encoding='utf-8')\n",
    "\n",
    "print(len(rawdata))\n",
    "print(len(rawdata[rawdata['source'] == '新华社']))\n",
    "rawdata.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- 不平衡数据，新华社的数据占据中数据的 85% 以上"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(87054, 2)\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>source</th>\n",
       "      <th>content</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <td>0</td>\n",
       "      <td>快科技@http://www.kkj.cn/</td>\n",
       "      <td>此外，自本周（6月12日）起，除小米手机6等15款机型外，其余机型已暂停更新发布（含开发版/...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>1</td>\n",
       "      <td>快科技@http://www.kkj.cn/</td>\n",
       "      <td>骁龙835作为唯一通过Windows 10桌面平台认证的ARM处理器，高通强调，不会因为只考...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>2</td>\n",
       "      <td>快科技@http://www.kkj.cn/</td>\n",
       "      <td>此前的一加3T搭载的是3400mAh电池，DashCharge快充规格为5V/4A。\\r\\n...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>3</td>\n",
       "      <td>新华社</td>\n",
       "      <td>这是6月18日在葡萄牙中部大佩德罗冈地区拍摄的被森林大火烧毁的汽车。新华社记者张立云摄\\r\\n</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>4</td>\n",
       "      <td>深圳大件事</td>\n",
       "      <td>（原标题：44岁女子跑深圳约会网友被拒，暴雨中裸身奔走……）\\r\\n@深圳交警微博称：昨日清...</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                   source                                            content\n",
       "0  快科技@http://www.kkj.cn/  此外，自本周（6月12日）起，除小米手机6等15款机型外，其余机型已暂停更新发布（含开发版/...\n",
       "1  快科技@http://www.kkj.cn/  骁龙835作为唯一通过Windows 10桌面平台认证的ARM处理器，高通强调，不会因为只考...\n",
       "2  快科技@http://www.kkj.cn/  此前的一加3T搭载的是3400mAh电池，DashCharge快充规格为5V/4A。\\r\\n...\n",
       "3                     新华社    这是6月18日在葡萄牙中部大佩德罗冈地区拍摄的被森林大火烧毁的汽车。新华社记者张立云摄\\r\\n\n",
       "4                   深圳大件事  （原标题：44岁女子跑深圳约会网友被拒，暴雨中裸身奔走……）\\r\\n@深圳交警微博称：昨日清..."
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# 语料数据和标签\n",
    "def get_data(rawdata):\n",
    "    mask = rawdata['content'].notnull()\n",
    "    data = rawdata.loc[mask, ['source', 'content']]\n",
    "    return data\n",
    "\n",
    "\n",
    "data = get_data(rawdata)\n",
    "print(data.shape)\n",
    "\n",
    "data.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "count    87054.000000\n",
       "mean       444.887380\n",
       "std        670.024847\n",
       "min          3.000000\n",
       "25%        132.000000\n",
       "50%        176.000000\n",
       "75%        499.000000\n",
       "max      22422.000000\n",
       "Name: content, dtype: float64"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# 预料数据的统计信息\n",
    "data.content.str.len().describe()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'此外，自本周（6月12日）起，除小米手机6等15款机型外，其余机型已暂停更新发布（含开发版/体验版内测，稳定版暂不受影响），以确保工程师可以集中全部精力进行系统优化工作。有人猜测这也是将精力主要用到MIUI 9的研发之中。\\r\\nMIUI 8去年5月发布，距今已有一年有余，也是时候更新换代了。\\r\\n当然，关于MIUI 9的确切信息，我们还是等待官方消息。\\r\\n'"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data.content[0]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'\\\\n\\\\n2017年5月25日，在美国马萨诸塞州剑桥市，哈佛大学毕业生在毕业典礼上欢呼。（新华/欧新）;;;;新华社北京6月7日电\\\\n（\\\\n记者\\\\n夏文辉）\\\\n据美国哈佛大学的知名校园媒体《哈佛深红色》消息，哈佛大学取消了至少10名新生的入学资格，因为他们在社交媒体脸书上发表了涉及性、种族歧视、儿童虐待及极端主义等不当内容。美联社等媒体5日转引了这一报道。\\\\n据报道，这些学生去年12月在脸书上建立了一个群，用于发布和共享信息，一些内容不堪入目。比如，一篇文章转载了一个死于叙利亚轰炸的儿童的血腥照片，图片不仅没有打码，不少学生还对此恶意戏谑。\\\\n据《哈佛深红色》报道，今年4月，哈佛大学招生部门向部分学生发函，要求他们解释在社交媒体上发布的极端和不当内容，并表示要审核他们的入学情况。校方同时通知他们不必参加4月新生的到校访问活动。大约一周后，至少10名学生接到通知，他们的入学资格被哈佛大学取消。\\\\n《哈佛深红色》没有给出未能入学学生的名字。美联社也未能联系到具体学生。哈佛大学发言人理查德·戴恩回复路透社采访询问说，校方不会公开讨论申请入学者情况。\\\\n按照哈佛大学的规定，以下几种情况有可能被学校取消入学资格：高中未能毕业、入学申请造假，以及申请人存在有违正直、诚实等品行的情况。\\\\n'"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data.content[89609]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "2557\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>id</th>\n",
       "      <th>author</th>\n",
       "      <th>source</th>\n",
       "      <th>content</th>\n",
       "      <th>feature</th>\n",
       "      <th>title</th>\n",
       "      <th>url</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <td>100</td>\n",
       "      <td>89517</td>\n",
       "      <td>NaN</td>\n",
       "      <td>中国证券报?中证网</td>\n",
       "      <td>NaN</td>\n",
       "      <td>{\"type\":\"公司\",\"site\":\"中证网\",\"commentNum\":\"0\",\"jo...</td>\n",
       "      <td>天和防务股东未来6个月内计划减持不超过480万股公司股份</td>\n",
       "      <td>http://www.cs.com.cn/ssgs/gsxw/201706/t2017062...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>103</td>\n",
       "      <td>89514</td>\n",
       "      <td>NaN</td>\n",
       "      <td>中国证券报?中证网</td>\n",
       "      <td>NaN</td>\n",
       "      <td>{\"type\":\"公司\",\"site\":\"中证网\",\"commentNum\":\"0\",\"jo...</td>\n",
       "      <td>晶盛机电调整限制性股票回购价格</td>\n",
       "      <td>http://www.cs.com.cn/ssgs/gsxw/201706/t2017062...</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "        id author     source content  \\\n",
       "100  89517    NaN  中国证券报?中证网     NaN   \n",
       "103  89514    NaN  中国证券报?中证网     NaN   \n",
       "\n",
       "                                               feature  \\\n",
       "100  {\"type\":\"公司\",\"site\":\"中证网\",\"commentNum\":\"0\",\"jo...   \n",
       "103  {\"type\":\"公司\",\"site\":\"中证网\",\"commentNum\":\"0\",\"jo...   \n",
       "\n",
       "                            title  \\\n",
       "100  天和防务股东未来6个月内计划减持不超过480万股公司股份   \n",
       "103               晶盛机电调整限制性股票回购价格   \n",
       "\n",
       "                                                   url  \n",
       "100  http://www.cs.com.cn/ssgs/gsxw/201706/t2017062...  \n",
       "103  http://www.cs.com.cn/ssgs/gsxw/201706/t2017062...  "
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# 无效的数据\n",
    "null_data = rawdata.loc[rawdata.content.isnull()]\n",
    "print(len(null_data))\n",
    "null_data.head(2)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 数据处理及向量化"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [],
   "source": [
    "def preprocessor(doc):\n",
    "    \"\"\"删除文本中所有非中文词及常用标点\"\"\"\n",
    "    regx = re.compile('[^\\u4e00-\\u9fa5‘’“”，、；。？！《》（）.0123456789%// A-Za-z]')\n",
    "    doc = regx.sub('', doc)\n",
    "    return doc\n",
    "\n",
    "\n",
    "def data2matrix(data):\n",
    "    \"\"\"文本向量化\"\"\"\n",
    "    y = data['source'].apply(lambda s: 1 if s == \"新华社\" else 0)\n",
    "    vectorizer = TfidfVectorizer(tokenizer=jieba.cut,\n",
    "                                 preprocessor=preprocessor,\n",
    "                                 max_df=0.8,\n",
    "                                 max_features=10000)\n",
    "    X = vectorizer.fit_transform(data['content'])\n",
    "    return X, y, vectorizer"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Building prefix dict from the default dictionary ...\n",
      "Dumping model to file cache /tmp/jieba.cache\n",
      "Loading model cost 0.412 seconds.\n",
      "Prefix dict has been built successfully.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "CPU times: user 1min 27s, sys: 123 ms, total: 1min 27s\n",
      "Wall time: 1min 27s\n"
     ]
    }
   ],
   "source": [
    "%%time\n",
    "X, y, vectorier = data2matrix(data)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Accuracy for AlwaysXinHuaShe 0.9035885772049532\n"
     ]
    }
   ],
   "source": [
    "# 不平衡数据，不论什么新闻总是判断为新华社，精度也达到 90%\n",
    "from sklearn.base import BaseEstimator\n",
    "\n",
    "\n",
    "class AlwaysXinHuaShe(BaseEstimator):\n",
    "    def fit(self, x, y=None):\n",
    "        pass\n",
    "\n",
    "    def predict(self, x):\n",
    "        return (np.ones((len(x), 1), dtype=bool))\n",
    "\n",
    "\n",
    "print(\"Accuracy for AlwaysXinHuaShe {}\".format(len(y[y == 1.]) / len(y)))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'此外': 6070,\n",
       " '自': 8026,\n",
       " '本周': 5752,\n",
       " '起': 8818,\n",
       " '除': 9566,\n",
       " '小米': 3781,\n",
       " '手机': 4713,\n",
       " '外': 3136,\n",
       " '其余': 1577,\n",
       " '已': 3978,\n",
       " '暂停': 5596,\n",
       " '更新': 5618,\n",
       " '发布': 2383,\n",
       " '含': 2634,\n",
       " '开发': 4228,\n",
       " '版': 6711,\n",
       " '影响': 4362,\n",
       " '以': 989,\n",
       " '确保': 7241,\n",
       " '工程师': 3954,\n",
       " '可以': 2485,\n",
       " '集中': 9613,\n",
       " '全部': 1481,\n",
       " '精力': 7595,\n",
       " '进行': 9100,\n",
       " '工作': 3936,\n",
       " '有人': 5668,\n",
       " '猜测': 6784,\n",
       " '这': 9050,\n",
       " '也': 712,\n",
       " '是': 5553,\n",
       " '将': 3751,\n",
       " '主要': 643,\n",
       " '研发': 7219,\n",
       " '之中': 668,\n",
       " '去年': 2302,\n",
       " '一年': 61,\n",
       " '时候': 5488,\n",
       " '了': 741,\n",
       " '当然': 4341,\n",
       " '关于': 1556,\n",
       " '信息': 1289,\n",
       " '我们': 4642,\n",
       " '还是': 9047,\n",
       " '等待': 7525,\n",
       " '作为': 1178,\n",
       " '唯一': 2730,\n",
       " '通过': 9210,\n",
       " '平台': 4089,\n",
       " '认证': 8501,\n",
       " '强调': 4315,\n",
       " '不会': 289,\n",
       " '因为': 2815,\n",
       " '只': 2475,\n",
       " '考虑': 7921,\n",
       " '性能': 4481,\n",
       " '而': 7926,\n",
       " '去': 2300,\n",
       " '小': 3758,\n",
       " '核心': 5940,\n",
       " '相反': 7113,\n",
       " '他们': 975,\n",
       " '正': 6050,\n",
       " '联手': 7960,\n",
       " '微软': 4414,\n",
       " '找到': 4789,\n",
       " '一种': 101,\n",
       " '适合': 9155,\n",
       " '、': 4,\n",
       " '兼顾': 1598,\n",
       " '和': 2687,\n",
       " '完美': 3578,\n",
       " '方案': 5393,\n",
       " '报道': 4865,\n",
       " '称': 7361,\n",
       " '已经': 3981,\n",
       " '拿到': 4943,\n",
       " '一些': 16,\n",
       " '新': 5339,\n",
       " '以便': 993,\n",
       " '更好': 5615,\n",
       " '地': 2939,\n",
       " '理解': 6873,\n",
       " '架构': 5893,\n",
       " '资料': 8755,\n",
       " '显示': 5561,\n",
       " '一款': 89,\n",
       " '集成': 9618,\n",
       " '比': 6127,\n",
       " '传统': 1093,\n",
       " '节省': 8147,\n",
       " '至少': 8072,\n",
       " '空间': 7404,\n",
       " '按计划': 4973,\n",
       " '今年': 947,\n",
       " '联想': 7959,\n",
       " '首发': 9813,\n",
       " '电脑': 6970,\n",
       " '预计': 9735,\n",
       " '均': 2981,\n",
       " '形态': 4349,\n",
       " '产品': 864,\n",
       " '只是': 2477,\n",
       " '个': 480,\n",
       " '开始': 4234,\n",
       " '未来': 5744,\n",
       " '也许': 714,\n",
       " '还': 9044,\n",
       " '能': 8004,\n",
       " '见到': 8428,\n",
       " '三星': 197,\n",
       " '华为': 2153,\n",
       " '澎湃': 6599,\n",
       " '等': 7519,\n",
       " '进入': 9085,\n",
       " '此前': 6068,\n",
       " '搭载': 5100,\n",
       " '电池': 6967,\n",
       " '规格': 8447,\n",
       " '为': 605,\n",
       " '至于': 8069,\n",
       " '可能': 2492,\n",
       " '与': 350,\n",
       " '虎': 8307,\n",
       " '所说': 4704,\n",
       " '要': 8419,\n",
       " '做': 1341,\n",
       " '最': 5635,\n",
       " '设定': 8539,\n",
       " '有关': 5671,\n",
       " '按照': 4971,\n",
       " '目前': 7089,\n",
       " '掌握': 5006,\n",
       " '拥有': 4929,\n",
       " '摄像头': 5106,\n",
       " '量': 9359,\n",
       " '“': 2,\n",
       " '惊喜': 4555,\n",
       " '”': 3,\n",
       " '根据': 5945,\n",
       " '京东': 876,\n",
       " '泄露': 6301,\n",
       " '售价': 2728,\n",
       " '元': 1375,\n",
       " '应该': 4176,\n",
       " '中': 489,\n",
       " '某个': 5896,\n",
       " '葡萄牙': 8276,\n",
       " '中部': 586,\n",
       " '大': 3213,\n",
       " '地区': 2946,\n",
       " '拍摄': 4908,\n",
       " '被': 8375,\n",
       " '森林': 5992,\n",
       " '汽车': 6243,\n",
       " '记者': 8520,\n",
       " '原': 2278,\n",
       " '标题': 5925,\n",
       " '岁': 3903,\n",
       " '女子': 3411,\n",
       " '跑': 8855,\n",
       " '深圳': 6472,\n",
       " '网友': 7824,\n",
       " '暴雨': 5606,\n",
       " '交警': 848,\n",
       " '昨日': 5552,\n",
       " '清晨': 6499,\n",
       " '发现': 2394,\n",
       " '有': 5665,\n",
       " '一': 7,\n",
       " '行走': 8341,\n",
       " '快速': 4461,\n",
       " '上': 207,\n",
       " '期间': 5736,\n",
       " '后': 2611,\n",
       " '赶紧': 8817,\n",
       " '其': 1574,\n",
       " '并': 4126,\n",
       " '一路': 122,\n",
       " '她': 3426,\n",
       " '那么': 9262,\n",
       " '事发': 760,\n",
       " '时': 5485,\n",
       " '到底': 1884,\n",
       " '都': 9296,\n",
       " '发生': 2396,\n",
       " '些': 825,\n",
       " '什么': 938,\n",
       " '呢': 2661,\n",
       " '？': 9999,\n",
       " '南': 2187,\n",
       " '带': 4042,\n",
       " '您': 4539,\n",
       " '一起': 121,\n",
       " '还原': 9046,\n",
       " '现场': 6832,\n",
       " '大队': 3302,\n",
       " '中队': 591,\n",
       " '女生': 3419,\n",
       " '一位': 21,\n",
       " '高大': 9905,\n",
       " '说话': 8620,\n",
       " '青年': 9654,\n",
       " '介绍': 954,\n",
       " '早上': 5475,\n",
       " '时分': 5490,\n",
       " '他': 973,\n",
       " '正在': 6053,\n",
       " '附近': 9530,\n",
       " '接到': 5026,\n",
       " '机动车': 5787,\n",
       " '危险': 2251,\n",
       " '随后': 9582,\n",
       " '寻找': 3738,\n",
       " '大概': 3264,\n",
       " '花': 8158,\n",
       " '大道': 3297,\n",
       " '出口': 1732,\n",
       " '往': 4370,\n",
       " '方向': 5391,\n",
       " '该': 8601,\n",
       " '身上': 8893,\n",
       " '停': 1349,\n",
       " '坐': 2987,\n",
       " '躺': 8906,\n",
       " '好': 3428,\n",
       " '另外': 2471,\n",
       " '一名': 36,\n",
       " '员': 2659,\n",
       " '追': 9138,\n",
       " '情绪': 4551,\n",
       " '很': 4383,\n",
       " '话': 8595,\n",
       " '不': 281,\n",
       " '多': 3164,\n",
       " '尝试': 3804,\n",
       " '交流': 843,\n",
       " '离开': 7303,\n",
       " '可': 2484,\n",
       " '愿意': 4600,\n",
       " '接受': 5028,\n",
       " '继续': 7769,\n",
       " '缓慢': 7798,\n",
       " '走': 8797,\n",
       " '此时': 6072,\n",
       " '路边': 8879,\n",
       " '聚集': 7974,\n",
       " '市民': 4016,\n",
       " '为了': 607,\n",
       " '刺激': 1911,\n",
       " '一边': 127,\n",
       " '着': 7174,\n",
       " '群众': 7888,\n",
       " '从': 957,\n",
       " '警方': 8487,\n",
       " '提供': 5059,\n",
       " '一份': 20,\n",
       " '视频': 8462,\n",
       " '了解': 742,\n",
       " '到': 1880,\n",
       " '出现': 1754,\n",
       " '监控': 7067,\n",
       " '穿着': 7409,\n",
       " '白色': 7044,\n",
       " '沿着': 6296,\n",
       " '当时': 4338,\n",
       " '正值': 6052,\n",
       " '上班': 240,\n",
       " '高峰期': 9914,\n",
       " '当': 4319,\n",
       " '路上': 8873,\n",
       " '纷纷': 7673,\n",
       " '观望': 8438,\n",
       " '不少': 311,\n",
       " '车辆': 8916,\n",
       " '速度': 9214,\n",
       " '但': 1115,\n",
       " '脚步': 8013,\n",
       " '依然': 1225,\n",
       " '行进': 8342,\n",
       " '中间': 589,\n",
       " '一辆': 125,\n",
       " '货车': 8713,\n",
       " '镜头': 9433,\n",
       " '但是': 1116,\n",
       " '没': 6262,\n",
       " '穿': 7406,\n",
       " '朝着': 5722,\n",
       " '周围': 2667,\n",
       " '没有': 6266,\n",
       " '人': 888,\n",
       " '或者': 4651,\n",
       " '看到': 7142,\n",
       " '这样': 9072,\n",
       " '情况': 4545,\n",
       " '恐怕': 4515,\n",
       " '大家': 3247,\n",
       " '办法': 1970,\n",
       " '面对': 9680,\n",
       " '这一': 9051,\n",
       " '表示': 8370,\n",
       " '根本': 5946,\n",
       " '不敢': 320,\n",
       " '看': 7137,\n",
       " '心里': 4439,\n",
       " '挺': 4982,\n",
       " '感觉': 4595,\n",
       " '尴尬': 3824,\n",
       " '跟随': 8865,\n",
       " '作出': 1179,\n",
       " '让': 8505,\n",
       " '举动': 651,\n",
       " '突然': 7417,\n",
       " '靠近': 9671,\n",
       " '上面': 255,\n",
       " '手': 4707,\n",
       " '控制': 5037,\n",
       " '住': 1129,\n",
       " '远离': 9106,\n",
       " '衣服': 8353,\n",
       " '把': 4810,\n",
       " '扔': 4751,\n",
       " '里': 9317,\n",
       " '只能': 2479,\n",
       " '紧紧': 7626,\n",
       " '拉': 4884,\n",
       " '一只': 32,\n",
       " '跟': 8861,\n",
       " '后面': 2619,\n",
       " '耐心': 7931,\n",
       " '听到': 2638,\n",
       " '不断': 321,\n",
       " '重复': 9335,\n",
       " '一句': 31,\n",
       " '要是': 8421,\n",
       " '你': 1191,\n",
       " '遭遇': 9250,\n",
       " '我': 4641,\n",
       " '事': 750,\n",
       " '会': 1067,\n",
       " '不时': 325,\n",
       " '试图': 8580,\n",
       " '就': 3810,\n",
       " '天': 3309,\n",
       " '大暴雨': 3259,\n",
       " '连': 9118,\n",
       " '眼睛': 7170,\n",
       " '说': 8617,\n",
       " '瞬间': 7190,\n",
       " '雨': 9627,\n",
       " '帮助': 4057,\n",
       " '冒': 1621,\n",
       " '大雨': 3304,\n",
       " '吧': 2632,\n",
       " '来说': 5858,\n",
       " '想': 4563,\n",
       " '回家': 2800,\n",
       " '然后': 6668,\n",
       " '照片': 6677,\n",
       " '旁边': 5405,\n",
       " '场景': 2976,\n",
       " '下来': 267,\n",
       " '带到': 4043,\n",
       " '派出所': 6367,\n",
       " '那': 9259,\n",
       " '姑娘': 3460,\n",
       " '什么样': 939,\n",
       " '事情': 765,\n",
       " '才': 4722,\n",
       " '据': 4995,\n",
       " '透露': 9180,\n",
       " '陈': 9536,\n",
       " '系': 7609,\n",
       " '湖北': 6544,\n",
       " '家属': 3681,\n",
       " '反映': 2364,\n",
       " '史': 2510,\n",
       " '三天': 192,\n",
       " '前': 1915,\n",
       " '陈某': 9538,\n",
       " '老家': 7901,\n",
       " '来': 5849,\n",
       " '导致': 3745,\n",
       " '异常': 4266,\n",
       " '产生': 869,\n",
       " '送往': 9153,\n",
       " '某': 5895,\n",
       " '医院': 2117,\n",
       " '治疗': 6292,\n",
       " '大大': 3239,\n",
       " '希望': 4038,\n",
       " '康复': 4193,\n",
       " '其实': 1579,\n",
       " '到来': 1886,\n",
       " '存在': 3503,\n",
       " '年龄': 4125,\n",
       " '限制': 9557,\n",
       " '你们': 1192,\n",
       " '因': 2814,\n",
       " '原因': 2286,\n",
       " '暖': 5601,\n",
       " '问': 9463,\n",
       " '这个': 9052,\n",
       " '哥哥': 2711,\n",
       " '吗': 2628,\n",
       " '一辈子': 126,\n",
       " '爱': 6694,\n",
       " '自己': 8036,\n",
       " '家人': 3676,\n",
       " '同时': 2589,\n",
       " '感谢': 4596,\n",
       " '聆听': 7937,\n",
       " '心灵': 4434,\n",
       " '点赞': 6642,\n",
       " '警察': 8483,\n",
       " '就是': 3815,\n",
       " '！': 9997,\n",
       " '男子': 6982,\n",
       " '号': 2518,\n",
       " '上午': 214,\n",
       " '公安局': 1507,\n",
       " '一个': 10,\n",
       " '报警': 4864,\n",
       " '电话': 6975,\n",
       " '声称': 3106,\n",
       " '遭到': 9248,\n",
       " '侵害': 1232,\n",
       " '几个': 1706,\n",
       " '关键词': 1568,\n",
       " '令': 987,\n",
       " '民警': 6171,\n",
       " '紧张': 7623,\n",
       " '起来': 8821,\n",
       " '受到': 2424,\n",
       " '股': 7980,\n",
       " '纳入': 7667,\n",
       " '指数': 4964,\n",
       " '利好': 1865,\n",
       " '消息': 6442,\n",
       " '市场': 4005,\n",
       " '周三': 2664,\n",
       " '再度': 1617,\n",
       " '上演': 239,\n",
       " '行情': 8334,\n",
       " '表现': 8369,\n",
       " '尾盘': 3841,\n",
       " '跳水': 8884,\n",
       " '之后': 672,\n",
       " '仅': 940,\n",
       " '金融': 9374,\n",
       " '板块': 5867,\n",
       " '仍': 955,\n",
       " '状态': 6770,\n",
       " '分析': 1789,\n",
       " '人士': 900,\n",
       " '认为': 8495,\n",
       " '受益': 2432,\n",
       " '于': 785,\n",
       " '估值': 1110,\n",
       " '资金': 8765,\n",
       " '青睐': 9662,\n",
       " '存量': 3506,\n",
       " '博弈': 2211,\n",
       " '格局': 5949,\n",
       " '下': 256,\n",
       " '风格': 9774,\n",
       " '震荡': 9643,\n",
       " '延续': 4199,\n",
       " '流动性': 6374,\n",
       " '改善': 5186,\n",
       " '经济': 7716,\n",
       " '预期': 9729,\n",
       " '有助于': 5676,\n",
       " '支撑': 5155,\n",
       " '大盘': 3276,\n",
       " '逐步': 9183,\n",
       " '向': 2620,\n",
       " '九': 709,\n",
       " '再现': 1619,\n",
       " '未能': 5747,\n",
       " '上行': 246,\n",
       " '态势': 4467,\n",
       " '两市': 454,\n",
       " '成交': 4617,\n",
       " '小幅': 3771,\n",
       " '一级': 109,\n",
       " '行业': 8326,\n",
       " '收盘': 5173,\n",
       " '银行': 9410,\n",
       " '两个': 433,\n",
       " '分别': 1778,\n",
       " '上涨': 235,\n",
       " '二级': 781,\n",
       " '来看': 5855,\n",
       " '涨幅': 6460,\n",
       " '最高': 5657,\n",
       " '达到': 8987,\n",
       " '信托': 1295,\n",
       " '及其': 2330,\n",
       " '保险': 1278,\n",
       " '证券': 8554,\n",
       " '成分股': 4622,\n",
       " '共有': 1550,\n",
       " '其中': 1575,\n",
       " '最大': 5644,\n",
       " '贵阳': 8742,\n",
       " '超过': 8833,\n",
       " '共': 1541,\n",
       " '个股': 488,\n",
       " '中国': 511,\n",
       " '居前': 3856,\n",
       " '两名': 440,\n",
       " '新华': 5351,\n",
       " '券商': 1909,\n",
       " '下跌': 275,\n",
       " '近期': 9037,\n",
       " '宣布': 3663,\n",
       " '年': 4105,\n",
       " '新兴': 5346,\n",
       " '占': 2216,\n",
       " '群体': 7889,\n",
       " '团队': 2828,\n",
       " '指出': 4956,\n",
       " '最新': 5649,\n",
       " '包含': 2054,\n",
       " '市值': 4002,\n",
       " '非': 9665,\n",
       " '互联互通': 798,\n",
       " '交易': 836,\n",
       " '股票': 7988,\n",
       " '以及': 998,\n",
       " '停牌': 1353,\n",
       " '标的': 5921,\n",
       " '由于': 6941,\n",
       " '很多': 4384,\n",
       " '权重': 5806,\n",
       " '由': 6940,\n",
       " '上升': 213,\n",
       " '消费': 6445,\n",
       " '涵盖': 6461,\n",
       " '大部分': 3298,\n",
       " '动态': 2013,\n",
       " '加入': 1980,\n",
       " '指': 4954,\n",
       " '增加': 3080,\n",
       " '其他': 1576,\n",
       " '尽管': 3839,\n",
       " '事件': 754,\n",
       " '对': 3715,\n",
       " '短期': 7203,\n",
       " '有所': 5683,\n",
       " '提振': 5067,\n",
       " '中长期': 588,\n",
       " '海外': 6414,\n",
       " '增量': 3094,\n",
       " '升温': 2137,\n",
       " '短期内': 7204,\n",
       " '尚': 3801,\n",
       " '不能': 338,\n",
       " '有效': 5684,\n",
       " '放大': 5208,\n",
       " '情景': 4550,\n",
       " '难以': 9595,\n",
       " '持续': 4948,\n",
       " '难': 9594,\n",
       " '改变': 5185,\n",
       " '并未': 4131,\n",
       " '引起': 4277,\n",
       " '热情': 6652,\n",
       " '成交量': 4618,\n",
       " '较为': 8954,\n",
       " '较大': 8956,\n",
       " '变化': 2441,\n",
       " '成长': 4640,\n",
       " '全天': 1461,\n",
       " '低迷': 1128,\n",
       " '表明': 8367,\n",
       " '很少': 4386,\n",
       " '不同': 304,\n",
       " '之间': 685,\n",
       " '使得': 1199,\n",
       " '突破': 7418,\n",
       " '中期': 561,\n",
       " '依旧': 1221,\n",
       " '维持': 7778,\n",
       " '虽然': 8313,\n",
       " '至今': 8070,\n",
       " '智能手机': 5595,\n",
       " '无法': 5425,\n",
       " '完全': 3571,\n",
       " '并于': 4129,\n",
       " '退出': 9147,\n",
       " '月份': 5660,\n",
       " '官方': 3588,\n",
       " '回归': 2802,\n",
       " '很快': 4388,\n",
       " '登场': 7030,\n",
       " '登陆': 7035,\n",
       " '又': 2327,\n",
       " '近日': 9036,\n",
       " '型号': 3006,\n",
       " '神秘': 7277,\n",
       " '悄然': 4531,\n",
       " '相关': 7112,\n",
       " '款': 6042,\n",
       " '并非': 4134,\n",
       " '定位': 3595,\n",
       " '所': 4695,\n",
       " '配备': 9300,\n",
       " '最受': 5641,\n",
       " '芯片': 8157,\n",
       " '之一': 665,\n",
       " '采用': 9311,\n",
       " '工艺': 3958,\n",
       " '设计': 8545,\n",
       " '当前': 4328,\n",
       " '只有': 2478,\n",
       " '上市': 223,\n",
       " '销售': 9417,\n",
       " '改进': 5192,\n",
       " '明显': 5525,\n",
       " '所以': 4696,\n",
       " '放在': 5207,\n",
       " '变成': 2443,\n",
       " '高端': 9932,\n",
       " '机': 5781,\n",
       " '不过': 347,\n",
       " '签署': 7540,\n",
       " '协议': 2170,\n",
       " '时间': 5510,\n",
       " '既然': 5440,\n",
       " '测试': 6386,\n",
       " '说明': 8618,\n",
       " '只要': 2480,\n",
       " '时期': 5497,\n",
       " '过': 8994,\n",
       " '新品': 5355,\n",
       " '之前': 671,\n",
       " '曝光': 5608,\n",
       " '图': 2904,\n",
       " '竞争': 7436,\n",
       " '优势': 1055,\n",
       " '全面': 1484,\n",
       " '推出': 5043,\n",
       " '全球': 1471,\n",
       " '首款': 9829,\n",
       " '多达': 3207,\n",
       " '媒体': 3488,\n",
       " '沟通': 6261,\n",
       " '会上': 1068,\n",
       " '各位': 2535,\n",
       " '时代': 5486,\n",
       " '成为': 4616,\n",
       " '沙漠': 6257,\n",
       " '有利': 5672,\n",
       " '消化': 6440,\n",
       " '开盘': 4253,\n",
       " '上证': 247,\n",
       " '带动': 4044,\n",
       " '一度': 63,\n",
       " '沪': 6269,\n",
       " '均线': 2983,\n",
       " '万达': 175,\n",
       " '电影': 6963,\n",
       " '绝大多数': 7757,\n",
       " '大幅': 3254,\n",
       " '各大': 2539,\n",
       " '全线': 1476,\n",
       " '不见': 343,\n",
       " '不可': 301,\n",
       " '策略': 7531,\n",
       " '题材': 9761,\n",
       " '为主': 606,\n",
       " '区域': 2098,\n",
       " '调整': 8643,\n",
       " '压力': 2274,\n",
       " '仍然': 956,\n",
       " '突出': 7412,\n",
       " '再次': 1618,\n",
       " '大跌': 3293,\n",
       " '处于': 3109,\n",
       " '底部': 4179,\n",
       " '低位': 1125,\n",
       " '反复': 2355,\n",
       " '夯实': 3346,\n",
       " '蓝筹股': 8296,\n",
       " '承压': 4793,\n",
       " '或': 4649,\n",
       " '后期': 2614,\n",
       " '需要': 9640,\n",
       " '接力': 5027,\n",
       " '走向': 8802,\n",
       " '今日': 951,\n",
       " '要闻': 8424,\n",
       " '央行': 3344,\n",
       " '上海': 231,\n",
       " '总部': 4509,\n",
       " '债券': 1324,\n",
       " '通': 9189,\n",
       " '北向': 2080,\n",
       " '境外': 3072,\n",
       " '投资者': 4837,\n",
       " '准入': 1693,\n",
       " '备案': 3123,\n",
       " '业务': 401,\n",
       " '指引': 4960,\n",
       " '二': 769,\n",
       " '我国': 4644,\n",
       " '信用': 1296,\n",
       " '法规': 6323,\n",
       " '标准': 5913,\n",
       " '研究': 7220,\n",
       " '加快': 1990,\n",
       " '推进': 5050,\n",
       " '三': 178,\n",
       " '房地产': 4688,\n",
       " '研究院': 7226,\n",
       " '《': 5,\n",
       " '房贷利率': 4693,\n",
       " '楼市': 5997,\n",
       " '报告': 4857,\n",
       " '》': 6,\n",
       " '许多': 8531,\n",
       " '城市': 3021,\n",
       " '个人': 481,\n",
       " '利率': 1872,\n",
       " '程度': 7373,\n",
       " '四': 2778,\n",
       " '水污染': 6199,\n",
       " '修正案': 1305,\n",
       " '草案': 8215,\n",
       " '更大': 5614,\n",
       " '力度': 1960,\n",
       " '保护': 1266,\n",
       " '水': 6186,\n",
       " '环境': 6819,\n",
       " '五': 803,\n",
       " '结算': 7748,\n",
       " '开展': 4237,\n",
       " '一人': 17,\n",
       " '多户': 3190,\n",
       " '通知': 9199,\n",
       " '启动': 2642,\n",
       " '同一': 2580,\n",
       " '以上': 990,\n",
       " '账户': 8707,\n",
       " '按': 4970,\n",
       " '文件': 5297,\n",
       " '要求': 8422,\n",
       " '自月': 8041,\n",
       " '日后': 5445,\n",
       " '交易日': 841,\n",
       " '确认': 7246,\n",
       " '六': 1536,\n",
       " '十三': 2119,\n",
       " '末': 5748,\n",
       " '金融市场': 9378,\n",
       " '达': 8986,\n",
       " '万亿': 151,\n",
       " '七': 142,\n",
       " '宏观': 3580,\n",
       " '稳定': 7382,\n",
       " '双重': 2352,\n",
       " '因素': 2820,\n",
       " '鼓舞': 9986,\n",
       " '国际': 2892,\n",
       " '基金': 3051,\n",
       " '经理': 7725,\n",
       " '下半年': 261,\n",
       " '股市': 7985,\n",
       " '有望': 5687,\n",
       " '十年': 2129,\n",
       " '增长': 3095,\n",
       " '创新': 1844,\n",
       " '驱动': 9867,\n",
       " '回顾': 2813,\n",
       " '早盘': 5484,\n",
       " '回落': 2810,\n",
       " '走势': 8801,\n",
       " '盘面': 7080,\n",
       " '深': 6465,\n",
       " '盘中': 7079,\n",
       " '新高': 5386,\n",
       " '分化': 1779,\n",
       " '相当': 7118,\n",
       " '至': 8068,\n",
       " '报点': 4861,\n",
       " '较前': 8955,\n",
       " '跌幅': 8854,\n",
       " '；': 9998,\n",
       " '创业板': 1835,\n",
       " '弱': 4295,\n",
       " '品种': 2699,\n",
       " '计算': 8492,\n",
       " '未': 5742,\n",
       " '涨停': 6459,\n",
       " '不足': 346,\n",
       " '亿元': 932,\n",
       " '增多': 3082,\n",
       " '数据': 5279,\n",
       " '主力': 623,\n",
       " '尽': 3835,\n",
       " '流出': 6372,\n",
       " '较': 8953,\n",
       " '一日': 74,\n",
       " '有个': 5666,\n",
       " '近': 9028,\n",
       " '介入': 953,\n",
       " '医药': 2116,\n",
       " '制造': 1903,\n",
       " '雄安': 9605,\n",
       " '新区': 5350,\n",
       " '更是': 5619,\n",
       " '高': 9895,\n",
       " '技术': 4804,\n",
       " '临近': 600,\n",
       " '半个': 2144,\n",
       " '小时': 3776,\n",
       " '高位': 9899,\n",
       " '留下': 6999,\n",
       " '以来': 1002,\n",
       " '每次': 6122,\n",
       " '回调': 2811,\n",
       " '相对': 7116,\n",
       " '位置': 1121,\n",
       " '即使': 2253,\n",
       " '乏力': 694,\n",
       " '总体': 4486,\n",
       " '打破': 4744,\n",
       " '攻克': 5196,\n",
       " '却': 2258,\n",
       " '当日': 4337,\n",
       " '无疑': 5428,\n",
       " '心理': 4435,\n",
       " '负面影响': 8689,\n",
       " '分钟': 1804,\n",
       " '线': 7680,\n",
       " '级别': 7654,\n",
       " '形成': 4350,\n",
       " '一旦': 75,\n",
       " '点': 6635,\n",
       " '意味着': 4572,\n",
       " '如此': 3449,\n",
       " '即': 2252,\n",
       " '引发': 4272,\n",
       " '综合': 7783,\n",
       " '处在': 3112,\n",
       " '区间': 2100,\n",
       " '脆弱': 8010,\n",
       " '心态': 4431,\n",
       " '碎片': 7248,\n",
       " '化': 2062,\n",
       " '模糊': 6012,\n",
       " '热点': 6654,\n",
       " '赚钱': 8772,\n",
       " '效应': 5238,\n",
       " '以下': 991,\n",
       " '首先': 9812,\n",
       " '长期': 9447,\n",
       " '人气': 921,\n",
       " '最近': 5655,\n",
       " '一段时间': 92,\n",
       " '整体': 5287,\n",
       " '走高': 8812,\n",
       " '基础': 3046,\n",
       " '弹性': 4299,\n",
       " '制约': 1899,\n",
       " '绝对': 7759,\n",
       " '大面积': 3305,\n",
       " '与此同时': 353,\n",
       " '现在': 6831,\n",
       " '非常': 9666,\n",
       " '频繁': 9756,\n",
       " '今天': 946,\n",
       " '明天': 5521,\n",
       " '就要': 3819,\n",
       " '打击': 4736,\n",
       " '积极性': 7358,\n",
       " '其次': 1580,\n",
       " '反弹': 2361,\n",
       " '重要': 9349,\n",
       " '区': 2092,\n",
       " '这种': 9075,\n",
       " '大涨': 3270,\n",
       " '似乎': 1114,\n",
       " '一直': 100,\n",
       " '吸引': 2648,\n",
       " '加码': 2002,\n",
       " '中小': 543,\n",
       " '面临': 9673,\n",
       " '强大': 4309,\n",
       " '午后': 2142,\n",
       " '收复': 5168,\n",
       " '投资': 4835,\n",
       " '建议': 4216,\n",
       " '始终': 3457,\n",
       " '缺乏': 7816,\n",
       " '支持': 5151,\n",
       " '背景': 7996,\n",
       " '后续': 2617,\n",
       " '若': 8180,\n",
       " '活跃': 6364,\n",
       " '创': 1832,\n",
       " '降低': 9547,\n",
       " '配置': 9302,\n",
       " '加大': 1984,\n",
       " '虽': 8312,\n",
       " '促使': 1246,\n",
       " '位': 1117,\n",
       " '能否': 8006,\n",
       " '真正': 7161,\n",
       " '观察': 8435,\n",
       " '基于': 3034,\n",
       " '组合': 7690,\n",
       " '因此': 2818,\n",
       " '股指': 7986,\n",
       " '获得': 8238,\n",
       " '再': 1615,\n",
       " '操作': 5140,\n",
       " '粤港澳': 7590,\n",
       " '大湾': 3271,\n",
       " '论坛': 8533,\n",
       " '腾讯': 8024,\n",
       " '人工智能': 906,\n",
       " '百度': 7050,\n",
       " '举办': 650,\n",
       " '开发者': 4231,\n",
       " '大会': 3218,\n",
       " '亚马逊': 824,\n",
       " '代表': 983,\n",
       " '互联网': 799,\n",
       " '巨头': 3968,\n",
       " '一致': 115,\n",
       " '看好': 7143,\n",
       " '发展': 2378,\n",
       " '产业': 857,\n",
       " '叠加': 2451,\n",
       " '政策': 5227,\n",
       " '扶持': 4780,\n",
       " '度': 4186,\n",
       " '一个月': 12,\n",
       " '内': 1599,\n",
       " '三次': 200,\n",
       " '加之': 1977,\n",
       " '月末': 5663,\n",
       " '值得': 1329,\n",
       " '做好': 1344,\n",
       " '应对': 4170,\n",
       " '月初': 5661,\n",
       " '为首': 615,\n",
       " '模式': 6008,\n",
       " '保持': 1269,\n",
       " '随着': 9586,\n",
       " '带来': 4049,\n",
       " '刷新': 1907,\n",
       " '高点': 9929,\n",
       " '降温': 9551,\n",
       " '跌': 8853,\n",
       " '加剧': 1982,\n",
       " '现象': 6840,\n",
       " '必须': 4443,\n",
       " '重视': 9352,\n",
       " '红利': 7637,\n",
       " '军工': 1630,\n",
       " '上周': 219,\n",
       " '荣耀': 8218,\n",
       " '类似': 7585,\n",
       " '来自': 5856,\n",
       " '比较': 6142,\n",
       " '敏感': 5244,\n",
       " '既': 5438,\n",
       " '半年': 2149,\n",
       " '容易': 3692,\n",
       " '实际上': 3631,\n",
       " '美联储': 7882,\n",
       " '加息': 1991,\n",
       " '行为': 8328,\n",
       " '货币政策': 8709,\n",
       " '开启': 4232,\n",
       " '流行': 6379,\n",
       " '很难': 4390,\n",
       " '式': 4269,\n",
       " '常态': 4069,\n",
       " '基本面': 3044,\n",
       " '这是': 9070,\n",
       " '阶段': 9505,\n",
       " '一段': 91,\n",
       " '高速': 9942,\n",
       " '增速': 3093,\n",
       " '下滑': 270,\n",
       " '而是': 7929,\n",
       " '周期': 2670,\n",
       " '适度': 9158,\n",
       " '供给': 1217,\n",
       " '侧': 1230,\n",
       " '结构性': 7744,\n",
       " '改革': 5194,\n",
       " '决定': 1680,\n",
       " '市': 4000,\n",
       " '运行': 9023,\n",
       " ...}"
      ]
     },
     "execution_count": 24,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "vectorier.vocabulary_"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 构建机器学习模型"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- 拆分数据集为 train、test"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "((65290, 10000), (21764, 10000))"
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "x_train, x_test, y_train, y_test = train_test_split(X, y) \n",
    "\n",
    "x_train.shape,x_test.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.9036759074896615"
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "len(y_train[y_train==1])/len(y_train)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## **`KNN`分类器**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [],
   "source": [
    "def build_model(clf, x, y, params):\n",
    "    model = GridSearchCV(clf, cv=3, param_grid=params)\n",
    "    model.fit(x, y)\n",
    "    return model\n",
    "\n",
    "clf = KNeighborsClassifier()\n",
    "params = {'n_neighbors':range(2,5), 'weights':('uniform', 'distance')}"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "CPU times: user 17min 26s, sys: 57.4 s, total: 18min 24s\n",
      "Wall time: 10min 43s\n"
     ]
    }
   ],
   "source": [
    "%%time\n",
    "model = build_model(clf, x_train, y_train, params)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "KNeighborsClassifier(algorithm='auto', leaf_size=30, metric='minkowski',\n",
       "                     metric_params=None, n_jobs=None, n_neighbors=2, p=2,\n",
       "                     weights='distance')"
      ]
     },
     "execution_count": 24,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "knn = model.best_estimator_\n",
    "knn"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "accuracy is:  0.8924370520124977\n",
      "precision is:  0.9883268482490273\n",
      "recall is:  0.8914547304170906\n",
      "f1_score is:  0.9373946995426953\n"
     ]
    }
   ],
   "source": [
    "def model_performance(model, x_test, y_test):\n",
    "    y_pred = model.predict(x_test)\n",
    "    print('accuracy is: ', accuracy_score(y_test, y_pred))\n",
    "    print('precision is: ', precision_score(y_test, y_pred))\n",
    "    print('recall is: ', recall_score(y_test, y_pred))\n",
    "    print('f1_score is: ', f1_score(y_test, y_pred))\n",
    "\n",
    "model_performance(knn, x_test, y_test)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- 模型分类的准确度为 0.941，即 94.1% 的文章分类正确\n",
    "- 查准率为0.946， 即模型判断为新华社的文章中，有 5.4% 是误分类\n",
    "- 查全率为0.991， 即所有新华社的文章，模型将其中的 0.9% 分类错误"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## **`SVM`分类**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "SVC(C=1.0, cache_size=200, class_weight=None, coef0=0.0,\n",
       "    decision_function_shape='ovr', degree=3, gamma='auto_deprecated',\n",
       "    kernel='rbf', max_iter=-1, probability=False, random_state=None,\n",
       "    shrinking=True, tol=0.001, verbose=False)"
      ]
     },
     "execution_count": 38,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "%%time\n",
    "svm_clf = SVC()\n",
    "svm_clf.fit(x_train, y_train)\n",
    "\n",
    "svm_clf"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "accuracy is:  0.9043374379709612\n",
      "precision is:  0.9043374379709612\n",
      "recall is:  1.0\n",
      "f1_score is:  0.9497659605269507\n"
     ]
    }
   ],
   "source": [
    "model_performance(svm_clf, x_test, y_test)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- `SVM`分类器精度较低，与全猜为新华社的效果相当；该分类器基本无效\n",
    "- 模型可以将新华社的文章全部判断正确\n",
    "- 原因：不均衡的训练数据？新华社远多于其它来源"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## **朴素贝叶斯分类**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%%time\n",
    "bayes_clf = GaussianNB()\n",
    "bayes_clf.fit(x_train, y_train)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "bayes_clf"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "model_performance(bayes_clf, x_test, y_test)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## **逻辑回归**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 47,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/home/student/anaconda3/envs/noam/lib/python3.7/site-packages/sklearn/linear_model/logistic.py:432: FutureWarning: Default solver will be changed to 'lbfgs' in 0.22. Specify a solver to silence this warning.\n",
      "  FutureWarning)\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "CPU times: user 3.13 s, sys: 92 ms, total: 3.22 s\n",
      "Wall time: 1.65 s\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "LogisticRegression(C=1.0, class_weight=None, dual=False, fit_intercept=True,\n",
       "                   intercept_scaling=1, l1_ratio=None, max_iter=100,\n",
       "                   multi_class='warn', n_jobs=None, penalty='l2',\n",
       "                   random_state=None, solver='warn', tol=0.0001, verbose=0,\n",
       "                   warm_start=False)"
      ]
     },
     "execution_count": 47,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "%%time\n",
    "logit_clf = LogisticRegression()\n",
    "logit_clf.fit(x_train, y_train)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 48,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "accuracy is:  0.969031428046315\n",
      "precision is:  0.9709146764443564\n",
      "recall is:  0.9955797175083833\n",
      "f1_score is:  0.9830925145494681\n"
     ]
    }
   ],
   "source": [
    "model_performance(logit_clf, x_test, y_test)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- 逻辑回归效果比较好，速度快"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## **随机森林**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/home/student/anaconda3/envs/noam/lib/python3.7/site-packages/sklearn/ensemble/forest.py:245: FutureWarning: The default value of n_estimators will change from 10 in version 0.20 to 100 in 0.22.\n",
      "  \"10 in version 0.20 to 100 in 0.22.\", FutureWarning)\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "CPU times: user 7.89 s, sys: 88 ms, total: 7.98 s\n",
      "Wall time: 7.64 s\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "RandomForestClassifier(bootstrap=True, class_weight=None, criterion='gini',\n",
       "                       max_depth=None, max_features='auto', max_leaf_nodes=None,\n",
       "                       min_impurity_decrease=0.0, min_impurity_split=None,\n",
       "                       min_samples_leaf=1, min_samples_split=2,\n",
       "                       min_weight_fraction_leaf=0.0, n_estimators=10,\n",
       "                       n_jobs=None, oob_score=False, random_state=None,\n",
       "                       verbose=0, warm_start=False)"
      ]
     },
     "execution_count": 25,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "%%time\n",
    "forest = RandomForestClassifier()\n",
    "forest.fit(x_train, y_train)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "accuracy is:  0.9676989523984562\n",
      "precision is:  0.9728074107276259\n",
      "recall is:  0.9920264093448451\n",
      "f1_score is:  0.9823229148331616\n"
     ]
    }
   ],
   "source": [
    "model_performance(forest, x_test, y_test)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- 速度较快\n",
    "- 效果还行"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 抄袭的文章判别\n",
    "- 测试文章中，模型判别为新华社，且概率越高，但同时该文章不属于新华社的，抄袭可能性越大"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 60,
   "metadata": {},
   "outputs": [],
   "source": [
    "indexes = y_test.index"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 61,
   "metadata": {},
   "outputs": [],
   "source": [
    "y_pred = forest.predict(x_test)\n",
    "y_pred_prob = forest.predict_proba(x_test)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 62,
   "metadata": {},
   "outputs": [],
   "source": [
    "y_pred_with_prob = pd.DataFrame({'label':y_pred,'prob':y_pred_prob[:,1]}, index=indexes)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 63,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>label</th>\n",
       "      <th>prob</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>59848</th>\n",
       "      <td>1</td>\n",
       "      <td>1.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6070</th>\n",
       "      <td>0</td>\n",
       "      <td>0.500000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4176</th>\n",
       "      <td>0</td>\n",
       "      <td>0.500000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>58593</th>\n",
       "      <td>1</td>\n",
       "      <td>1.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>48695</th>\n",
       "      <td>1</td>\n",
       "      <td>1.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>73288</th>\n",
       "      <td>1</td>\n",
       "      <td>0.900000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>72765</th>\n",
       "      <td>1</td>\n",
       "      <td>0.900000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3077</th>\n",
       "      <td>0</td>\n",
       "      <td>0.500000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>48236</th>\n",
       "      <td>1</td>\n",
       "      <td>1.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5019</th>\n",
       "      <td>0</td>\n",
       "      <td>0.400000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>71412</th>\n",
       "      <td>1</td>\n",
       "      <td>0.800000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>35784</th>\n",
       "      <td>1</td>\n",
       "      <td>1.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6968</th>\n",
       "      <td>0</td>\n",
       "      <td>0.500000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>72388</th>\n",
       "      <td>1</td>\n",
       "      <td>1.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>79593</th>\n",
       "      <td>1</td>\n",
       "      <td>1.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>69122</th>\n",
       "      <td>1</td>\n",
       "      <td>1.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>24880</th>\n",
       "      <td>1</td>\n",
       "      <td>1.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>16342</th>\n",
       "      <td>1</td>\n",
       "      <td>1.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>86174</th>\n",
       "      <td>1</td>\n",
       "      <td>1.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>34651</th>\n",
       "      <td>1</td>\n",
       "      <td>1.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>51126</th>\n",
       "      <td>1</td>\n",
       "      <td>1.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>37647</th>\n",
       "      <td>1</td>\n",
       "      <td>0.800000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>78329</th>\n",
       "      <td>1</td>\n",
       "      <td>1.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10765</th>\n",
       "      <td>0</td>\n",
       "      <td>0.400000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>12278</th>\n",
       "      <td>1</td>\n",
       "      <td>1.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>34039</th>\n",
       "      <td>1</td>\n",
       "      <td>1.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1023</th>\n",
       "      <td>1</td>\n",
       "      <td>0.864907</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>56415</th>\n",
       "      <td>1</td>\n",
       "      <td>1.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4640</th>\n",
       "      <td>0</td>\n",
       "      <td>0.100000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>42749</th>\n",
       "      <td>1</td>\n",
       "      <td>1.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>63280</th>\n",
       "      <td>1</td>\n",
       "      <td>1.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>83019</th>\n",
       "      <td>1</td>\n",
       "      <td>1.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>68667</th>\n",
       "      <td>1</td>\n",
       "      <td>0.800000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>21799</th>\n",
       "      <td>1</td>\n",
       "      <td>0.800000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>57730</th>\n",
       "      <td>1</td>\n",
       "      <td>1.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7812</th>\n",
       "      <td>0</td>\n",
       "      <td>0.300000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>50318</th>\n",
       "      <td>1</td>\n",
       "      <td>1.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>15699</th>\n",
       "      <td>1</td>\n",
       "      <td>0.900000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>82066</th>\n",
       "      <td>1</td>\n",
       "      <td>0.700000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>72339</th>\n",
       "      <td>1</td>\n",
       "      <td>1.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>82872</th>\n",
       "      <td>1</td>\n",
       "      <td>1.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>58491</th>\n",
       "      <td>1</td>\n",
       "      <td>1.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>30555</th>\n",
       "      <td>1</td>\n",
       "      <td>1.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10799</th>\n",
       "      <td>1</td>\n",
       "      <td>0.776578</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>75655</th>\n",
       "      <td>1</td>\n",
       "      <td>1.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>33337</th>\n",
       "      <td>0</td>\n",
       "      <td>0.500000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>87583</th>\n",
       "      <td>1</td>\n",
       "      <td>1.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7402</th>\n",
       "      <td>1</td>\n",
       "      <td>0.600000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>54475</th>\n",
       "      <td>1</td>\n",
       "      <td>1.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7414</th>\n",
       "      <td>0</td>\n",
       "      <td>0.200000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>68844</th>\n",
       "      <td>1</td>\n",
       "      <td>0.800000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>66463</th>\n",
       "      <td>1</td>\n",
       "      <td>1.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>461</th>\n",
       "      <td>1</td>\n",
       "      <td>0.600000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>70201</th>\n",
       "      <td>1</td>\n",
       "      <td>1.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>79873</th>\n",
       "      <td>1</td>\n",
       "      <td>1.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>53367</th>\n",
       "      <td>1</td>\n",
       "      <td>1.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>68259</th>\n",
       "      <td>1</td>\n",
       "      <td>1.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>19700</th>\n",
       "      <td>1</td>\n",
       "      <td>1.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>41342</th>\n",
       "      <td>1</td>\n",
       "      <td>1.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>83031</th>\n",
       "      <td>1</td>\n",
       "      <td>1.000000</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>21764 rows × 2 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "       label  prob\n",
       "59848      1   1.0\n",
       "6070       0   0.5\n",
       "4176       0   0.5\n",
       "58593      1   1.0\n",
       "48695      1   1.0\n",
       "...      ...   ...\n",
       "53367      1   1.0\n",
       "68259      1   1.0\n",
       "19700      1   1.0\n",
       "41342      1   1.0\n",
       "83031      1   1.0\n",
       "\n",
       "[21764 rows x 2 columns]"
      ]
     },
     "execution_count": 63,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "y_pred_with_prob"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 65,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Int64Index([ 2473,  4993,  5394,  5197,  4539,  6001,  7559,  2974,  4565,\n",
       "            86202, 86565,  7037,  5174, 45999,  4106,  5949,  6612,   453,\n",
       "              670,  5309],\n",
       "           dtype='int64')"
      ]
     },
     "execution_count": 65,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "mask1 = y_pred_with_prob[(y_pred_with_prob.label==1.0)&(y_pred_with_prob.prob<0.60)].index\n",
    "mask1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 66,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Int64Index([ 6070,  4176,  3077,  5019,  6968, 10765,  1023,  4640,   407,\n",
       "             6446,\n",
       "            ...\n",
       "             7349,  3126,  4995,     4,  1881,  4184, 10799,  7402,  7414,\n",
       "              461],\n",
       "           dtype='int64', length=2074)"
      ]
     },
     "execution_count": 66,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "mask2 = y_test[y_test==0].index\n",
    "mask2"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 68,
   "metadata": {
    "scrolled": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "16"
      ]
     },
     "execution_count": 68,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "plagiaristic = data.loc[mask1.intersection(mask2)]\n",
    "len(plagiaristic)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 69,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>source</th>\n",
       "      <th>content</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>4993</th>\n",
       "      <td>广州日报第ZSA15版</td>\n",
       "      <td>中山分类信息\\r\\n</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5394</th>\n",
       "      <td>广州日报第A20版</td>\n",
       "      <td>对联\\r\\n　　上联：爱妻，爱子，爱家庭，不爱身体等于零；\\r\\n　　下联：有钱，有权，有成...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5197</th>\n",
       "      <td>广州日报第FS1版</td>\n",
       "      <td>霍震霆： 我的孩子 特别爱国\\r\\n　　8\\r\\n</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4539</th>\n",
       "      <td>广州日报第A24版</td>\n",
       "      <td>广州日报大收订\\r\\n</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6001</th>\n",
       "      <td>海南日报第010版</td>\n",
       "      <td>海南义隆实业有限公司：\\r\\n　　纳税人识别号：460100730064372\\r\\n　...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7559</th>\n",
       "      <td>广州日报第01版</td>\n",
       "      <td>无人驾驶来 道路将咋变\\r\\n</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2974</th>\n",
       "      <td>证券时报?e公司</td>\n",
       "      <td>6月15日，央行上海总部公布统计数据显示，5月上海地区个人住房贷款新增154.86亿元，...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4565</th>\n",
       "      <td>央视网</td>\n",
       "      <td>央视网消息：防治土壤污染，直接关系到农产品质量安全、人民群众身体健康和经济社会的可持续发展。...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7037</th>\n",
       "      <td>Wind资</td>\n",
       "      <td>Wind资讯，周四（6月22日），金融股带头，地产股跟风，沪深两市股指涨幅不断扩大，上证...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5174</th>\n",
       "      <td>广州日报第FS1版</td>\n",
       "      <td>新会出土 唐胡人陶俑\\r\\n　　10\\r\\n</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4106</th>\n",
       "      <td>广州日报第ZSA15版</td>\n",
       "      <td>中山分类信息\\r\\n</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5949</th>\n",
       "      <td>广州日报第02版</td>\n",
       "      <td>周末闷热多雷雨\\r\\n　　广州天气\\r\\n　　今天\\r\\n　　多云，午后有雷阵雨\\r\\n　　...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6612</th>\n",
       "      <td>广州日报第01版</td>\n",
       "      <td>霍震霆： 我的孩子 特别爱国\\r\\n　　8\\r\\n</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>453</th>\n",
       "      <td>solidot@</td>\n",
       "      <td>但 SMBv1 协议实现的漏洞导致了勒索软件 WannaCry 在存在漏洞的系统中广泛传播，...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>670</th>\n",
       "      <td>澎湃新闻网</td>\n",
       "      <td>香火钱”是香客捐给寺庙，用于日常供奉香烛等，然而，一些不法分子表面佯装虔诚的信徒，进庙烧香拜...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5309</th>\n",
       "      <td>广州日报第DG1版</td>\n",
       "      <td>新会出土 唐胡人陶俑\\r\\n　　10\\r\\n</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "           source                                            content\n",
       "4993  广州日报第ZSA15版                                         中山分类信息\\r\\n\n",
       "5394    广州日报第A20版  对联\\r\\n　　上联：爱妻，爱子，爱家庭，不爱身体等于零；\\r\\n　　下联：有钱，有权，有成...\n",
       "5197    广州日报第FS1版                          霍震霆： 我的孩子 特别爱国\\r\\n　　8\\r\\n\n",
       "4539    广州日报第A24版                                        广州日报大收订\\r\\n\n",
       "6001    海南日报第010版  　　海南义隆实业有限公司：\\r\\n　　纳税人识别号：460100730064372\\r\\n　...\n",
       "7559     广州日报第01版                                    无人驾驶来 道路将咋变\\r\\n\n",
       "2974     证券时报?e公司  　　6月15日，央行上海总部公布统计数据显示，5月上海地区个人住房贷款新增154.86亿元，...\n",
       "4565          央视网  央视网消息：防治土壤污染，直接关系到农产品质量安全、人民群众身体健康和经济社会的可持续发展。...\n",
       "7037        Wind资  　　Wind资讯，周四（6月22日），金融股带头，地产股跟风，沪深两市股指涨幅不断扩大，上证...\n",
       "5174    广州日报第FS1版                             新会出土 唐胡人陶俑\\r\\n　　10\\r\\n\n",
       "4106  广州日报第ZSA15版                                         中山分类信息\\r\\n\n",
       "5949     广州日报第02版  周末闷热多雷雨\\r\\n　　广州天气\\r\\n　　今天\\r\\n　　多云，午后有雷阵雨\\r\\n　　...\n",
       "6612     广州日报第01版                          霍震霆： 我的孩子 特别爱国\\r\\n　　8\\r\\n\n",
       "453      solidot@  但 SMBv1 协议实现的漏洞导致了勒索软件 WannaCry 在存在漏洞的系统中广泛传播，...\n",
       "670         澎湃新闻网  香火钱”是香客捐给寺庙，用于日常供奉香烛等，然而，一些不法分子表面佯装虔诚的信徒，进庙烧香拜...\n",
       "5309    广州日报第DG1版                             新会出土 唐胡人陶俑\\r\\n　　10\\r\\n"
      ]
     },
     "execution_count": 69,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "plagiaristic"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- 部分文章字数太少，无从确定是否抄袭"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 71,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "4993      8\n",
       "5394    127\n",
       "5197     21\n",
       "4539      9\n",
       "6001    567\n",
       "7559     13\n",
       "2974    151\n",
       "4565    114\n",
       "7037     97\n",
       "5174     18\n",
       "4106      8\n",
       "5949     93\n",
       "6612     21\n",
       "453     164\n",
       "670     349\n",
       "5309     18\n",
       "Name: content, dtype: int64"
      ]
     },
     "execution_count": 71,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "plagiaristic['content'].str.len()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 72,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'\\u3000\\u3000海南义隆实业有限公司：\\r\\n\\u3000\\u3000纳税人识别号：460100730064372\\r\\n\\u3000\\u3000事由：你公司在规定期限内未履行已生效的《海南省地方税务局第一稽查局税务处理决定书》（琼地税一稽处〔2014〕22号）和《海南省地方税务局第一稽查局税务行政处罚决定书》（琼地税一稽罚〔2015〕3号）应缴纳的税费，经催缴仍未缴纳，根据《中华人民共和国税收征收管理法》第四十条规定，采取强制执行措施，扣缴税款20,588.00元，拍卖财产抵缴罚款68,150.00元。尚欠税款652,371.49元、滞纳金304,938.95元、罚款183,767.24元，合计1,141,077.68元（因超期限未缴纳税款产生的滞纳金以金三系统计算为准）。\\r\\n\\u3000\\u3000因采取其他方式无法送达，根据《中华人民共和国税收征收管理法实施细则》第一百零六条规定，现依法向你公司公告送达《催告通知书》（琼地税一稽催通〔2017〕2号）。自本公告发出之日起满30日，即视为送达。\\r\\n\\u3000\\u3000如不服该《催告通知书》，必须根据本催告的期限缴纳税款，然后可自上述款项缴清之日起六十日内依法向海南省地方税务局第一稽查局申请行政复议。逾期本《催告通知书》即发生法律效力。\\r\\n\\u3000\\u3000附件：《催告通知书》（琼地税一稽催通〔2017〕2号）\\r\\n\\u3000\\u3000海南省地方税务局第一稽查局???2017年6月23日\\r\\n'"
      ]
     },
     "execution_count": 72,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "text1 = plagiaristic.loc[6001]['content']\n",
    "text1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 73,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'香火钱”是香客捐给寺庙，用于日常供奉香烛等，然而，一些不法分子表面佯装虔诚的信徒，进庙烧香拜佛，背地里却盯上了放在案台上的香火钱。\\r\\n近日，上海市青浦公安抓获违法人员敬某，当场缴获其盗窃所得。\\r\\n6月23日，青浦区徐泾镇蟠龙庵举行庙会，赶会香客近万人。此时，一名中年女子混迹香客之中不停地来回转悠，目光还时不时盯向落在供奉菩萨案台上香客们留下的硬币。观察一阵后，该女子佯装拜佛祷告，夹杂在人群中接近案台，乘人不备迅速伸手抓取案台上的硬币放进自己的口袋。如此陆续没过几分钟，案台上的“香火钱”就被该女子悉数收入囊中。让她没想到的是，自己的盗窃行为已被庵内工作人员发现并报警。就在该女子发现情况不妙准备离开庵堂溜之大吉时，被及时赶到的民警抓获。\\r\\n目前，敬某因盗窃行为已被青浦公安分局依法行政拘留。\\r\\n'"
      ]
     },
     "execution_count": 73,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "text2 = plagiaristic.loc[670]['content']\n",
    "text2"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 77,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'但 SMBv1 协议实现的漏洞导致了勒索软件 WannaCry 在存在漏洞的系统中广泛传播，然而受影响的系统主要是 Windows 7 而不是微软计划更新的 Windows 10。\\r\\n微软证实，用户在全新安装系统时候将不会包含 SMBv1，但从现有系统升级时 SMBv1 仍然会留在系统中。微软的这一决定不会影响现有的系统。\\r\\n'"
      ]
     },
     "execution_count": 77,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "text3 = plagiaristic.loc[453]['content']\n",
    "text3"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "\n",
    "    "
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.7"
  },
  "toc": {
   "base_numbering": 1,
   "nav_menu": {},
   "number_sections": true,
   "sideBar": true,
   "skip_h1_title": false,
   "title_cell": "Table of Contents",
   "title_sidebar": "Contents",
   "toc_cell": false,
   "toc_position": {},
   "toc_section_display": true,
   "toc_window_display": true
  },
  "varInspector": {
   "cols": {
    "lenName": 16,
    "lenType": 16,
    "lenVar": 40
   },
   "kernels_config": {
    "python": {
     "delete_cmd_postfix": "",
     "delete_cmd_prefix": "del ",
     "library": "var_list.py",
     "varRefreshCmd": "print(var_dic_list())"
    },
    "r": {
     "delete_cmd_postfix": ") ",
     "delete_cmd_prefix": "rm(",
     "library": "var_list.r",
     "varRefreshCmd": "cat(var_dic_list()) "
    }
   },
   "types_to_exclude": [
    "module",
    "function",
    "builtin_function_or_method",
    "instance",
    "_Feature"
   ],
   "window_display": false
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
